Selective outcome reporting and the effectiveness of psychotherapies for depression

Only 40% of trials psychotherapies for depression publish­ed between 2015 and 2018 were prospectively registered, discrepancies publications protocols noted 76% registered trials1. It is often assumed that such divergences are the result intentionally favoring statistically significant findings (“selective reporting”). However, could be due to other reasons, as justified protocol amendments, logistic difficulties or carelessness. A survey published in high-impact clinical psychology journals over four years2 identified 27 trials, which only 13 with a clearly specified primary outcome­ measure time assessment. Among these contained deviations (for two others this was impossible adjudicate). it difficult reliably estimate prevalence impact selective reporting from investigations small cohorts trials. Therefore, we examined differences effectiveness associated across complete cohort depression. We conducted pre-registered (PROSPERO: CRD42019136130) all randomized comparing psychological interventions control conditions adult started enrollment after July 1, 2005, when journal registration mandates became widespread3. selected regularly updated meta-analysis (https://osf.io/825c6/), using most recent update (January 2020). matching registrations publication, key word searches public registries, or, failing these, by contact investigators. Registration considered prospective if occurred within one month start. For pre-specified outcome assessment point, changes outcomes registries publications. Potential included4: a) omission (non-­reporting); b) addition new, not outcome; c) downgrading secondary; d) upgrading secondary primary; e) point changes; f) analysis method changes. Selective adjudicated b), and, discrepancies, on basis judgement independent researchers. Effect sizes computed standardized mean (SMDs) intervention at post-treatment primary, data event (e.g., response, remission), odds ratios converted them into SMDs5. pooled effect separately without reporting, robust variance estimation weights random effects model, sample adjustment an correlation pairs observed 0.86. Analyses run Stata/SE 16.1. found that, out 353 controlled cohort, 185 commenced 2005. Of 142 (77%, 95% CI: 70%-83%) registered. Seventy-five (40%, 33%-48%) prospectively, 11 (15%, 8%-25%) specifying measures points. Fifty-one (68%, 56%-78%) rated free reporting. Discrepancies reports 19/75 (25%, 16%-37%) (17%, 10%-28%) judged involving six omitted outcome, queried investigators received replies, explaining had been dropped before starting collection. The summary size –0.81 (95% –1.25 –0.38, tau2=0.22) –0.54 –0.65 –0.43, tau2=0.10) without. When analyses limited slightly reduced –0.75 –1.21 –0.29). Conversely, excluding led considerably (SMD=–0.51, –0.83 –0.19), closely resembling Similarly, added non-registered (SMD=–0.62, –1.00 –0.24) Finally, restricted self-report unblinded showed substantially increased (SMD=–1.02, –1.66 –0.38), but minimal (SMD=–0.57, –0.69 –0.44). Our confirm prior smaller more circumscribed surveys1, 2, showing even many condition­ed submission registration, implemented 25% displayed publications, 17% statistical significance. Though relatively few, larger effectiveness, combined meta-analysis. diverged SMD 0.27 reference, publication has 0.327. Trials non-reporting ones emerged main drivers inflation. These suggest lack remain common depression, inflation those contributing current uncertainties assessing interventions8, 9. Supplementary information study available https://doi.org/10.17605/OSF.IO/NBYW8 Please note: publisher responsible content functionality any supporting supplied authors. Any queries (other than missing content) should directed corresponding author article.